1. Field of the Invention
The present invention relates to a technique for measuring the position and orientation of an object body.
2. Description of the Related Art
The technique relating to mixed reality can realize a seamless combination of a physical space and a virtual space. For example, to realize an intended mixed reality, an image display apparatus inputs an image of the physical space from a video camera and combines the obtained physical space image with a virtual space image (for example, including virtual object bodies and character information created according to the computer graphics) which can be produced with reference to the position and the orientation of the camera.
Such a mixed reality can be realized only when a relative position and a relative orientation between a base coordinate system and a camera coordinate system are accurately measurable. The base coordinate system is defined in the physical space as a standard coordinate system which can determine the position and the orientation of virtual object bodies to be combined.
More specifically, when a virtual object body (i.e., a virtual space image) is drawn at a predetermined position in the physical space, an image of this virtual object body must be created with camera parameters identical with the camera parameters conforming to the base coordinate system. For example, when a virtual object body is placed at a predetermined position (e.g., in a room or on a table) in the physical space, the base coordinate system can be defined at an appropriate place (e.g., on a floor or on a table) in the physical space. The position and the orientation of a camera can be obtained in the base coordinate system.
Furthermore, the mixed reality technique can combine any information on a real object body that is movable. For example, a virtual pattern or a label can be combined on a real box held by an observer. If the object body coordinate system of a combined object body (i.e., the box) is a base coordinate system, the position and the orientation of a camera can be obtained in the base coordinate system (i.e., in the object body coordinate system of the box).
To realize such a measurement, there is one method (referred to as a “first approach”). First, plural indices (e.g., markers or natural features) can be disposed or set on the surface of a box (i.e., an object body to be combined). Then, the coordinates of each projected index can be detected on an image obtained from a camera. Then, the position and the orientation of the box relative to the camera can be obtained based on the position of each detected index in the object body coordinate system.
There is another method (referred to as a “second approach”). First, a base coordinate system can be defined in the physical space. Then, positions and orientations of the camera and the box (i.e., an object body to be combined) can be obtained in the base coordinate system. Then, a relative position and a relative orientation between the camera and the box can be obtained based on their position and orientation data obtained in the common coordinate system. For example, an outside-in camera can be additionally provided in the physical space to obtain an image of the measuring objects (i.e., the camera and the box).
The coordinates of plural indices disposed or set on the camera and the box can be detected from the image obtained by the outside-in camera. Then, based on a relationship between the projected image of indices on the camera and the indices in the camera coordinate system, a position and an orientation of the camera in the base coordinate system can be calculated. Furthermore, based on a relationship between the projected image of an index on the object body and the index position in the object body coordinate system, a position and an orientation of the box can be calculated in the base coordinate system. Then, a position and an orientation of the box relative to the camera can be measured based on the position and orientation data of the camera and the box in the base coordinate system.
Furthermore, the second approach can be realized by applying a conventional method (such as a method for measuring the position and the orientation of a measuring object body in the base coordinate system) to measure positions and orientations of respective measuring object bodies. For example, plural indices can be disposed at known positions in the physical space, and image coordinates of respective indices can be detected from an image taken by a camera (i.e., a measuring object). The position and the orientation of the camera (i.e., the measuring object) in the base coordinate system can be obtained based on the detected image coordinates. Similarly, an inside-out camera can be mounted on an object body to be combined (i.e., a measuring object body). The position and the orientation of the object body in the base coordinate system can be obtained based on the indices imaged and detected by the inside-out camera.
Furthermore, a camera serving as a measuring object body (or an inside-out camera mounted on a measuring object body) can obtain the information relating to indices in a physical space, and an outside-in camera placed in a physical space can obtain the information relating to indices on the measuring object body. The information obtained by the measuring object camera (or the inside-out camera) and the information obtained by the outside-in camera can be used to obtain the position and orientation of the measuring object body in the base coordinate system (for example, refer to K. Satoh, S. Uchiyama, H. Yamamoto, and H. Tamura: “Robust vision-based registration utilizing bird's-eye view with user's view,” Proc. 2nd International Symposium on Mixed and Augmented Reality (ISMAR'03), pp. 46-55, 2003, and E. Foxlin, Y. Altshuler, L. Naimarkand M. Harrington: “FlightTracker: A novel optical/inertial tracker for cockpit enhanced vision,” Proc. 3rd International Symposium on Mixed and Augmented Reality (ISMAR '04), pp. 212-221, 2004).
However, according to the first approach, the index (indices) on a box (i.e., an object body to be combined) must be constantly and sufficiently observed by a camera (i.e., a measuring object). Therefore, if the surface of a box is covered with a hand of the observer, combining the information will be difficult. Furthermore, according to the first approach, positions and orientations of respective measuring object bodies in the base coordinate system cannot be obtained. The first approach cannot be used to display the measuring object bodies simultaneously with virtual information fixed in the base coordinate system.
Furthermore, according to the second approach, the information relating to an object body to be combined cannot be used in the measurement, even if such information can be obtained by a camera (i.e., a measuring object). Errors in the measurement of positions and orientations of the camera and a box (i.e., an object body to be combined) will appear as positional deviations of virtual object bodies on a combined image.